Sketches for Matrix Norms: Faster, Smaller and More General

We design new sketching algorithms for unitarily invariant matrix norms, including the Schatten p-norms ‖·‖Sp , and obtain, as a by-product, streaming algorithms that approximate the norm of a matrix A presented as a turnstile data stream. The primary advantage of our streaming algorithms is that they are simpler and faster than previous algorithms, while requiring the same or less storage. Our three main results are a faster sketch for estimating ‖A‖Sp , a smaller-space O(1)-pass sketch for ‖A‖Sp , and more general sketching technique that yields sublinear-space approximations for a wide class of matrix norms. These improvements are powered by dimensionality reduction techniques that are modern incarnations of the JohnsonLindenstrauss Lemma [JL84]. When p ≥ 2 is even or A is PSD, our fast one-pass algorithm approximates ‖A‖Sp in optimal, n, space with O(1) update time and o(n) time to extract the approximation from the sketch, while the ⌈p/2⌉-pass algorithm is built on a smaller sketch of size n with O(1) update time and n query time. Finally, for a PSD matrix A and a unitarily invariant norm l(·), we prove that one can obtain an approximation to l(A) from a sketch GAH where G and H are independent Oblivious Subspace Embeddings and the dimension of the sketch is polynomial in the intrinsic dimension of A. The intrinsic dimension of a matrix is a robust version of the rank that is equal to the ratio ∑ i σi/σ1. It is small, e.g., for models in machine learning which consist of a low rank matrix plus noise. Naturally, this leads to much smaller sketches for many norms. Email: [email protected]. This material is based upon work supported in part by the National Science Foundation under Grant No. 1447639, by the Google Faculty Award and by DARPA grant N660001-1-2-4014. Its contents are solely the responsibility of the authors and do not represent the official view of DARPA or the Department of Defense. Email: [email protected] Email: [email protected]. Work supported in part by the Israel Science Foundation grant #897/13. Email: [email protected]